Aerial Suspended Cargo Delivery through Reinforcement Learning

Cargo-bearing Unmanned aerial vehicles (UAVs) have tremendous potential to assist humans in food, medicine, and supply deliveries. For time-critical cargo delivery tasks, UAVs need to be able to navigate their environments and deliver suspended payloads with bounded load displacement. As a constraint balancing task for joint UAV-suspended load system dynamics, this task poses a challenge. This article presents a reinforcement learning approach to aerial cargo delivery tasks in environments with static obstacles. We first learn a minimal residual oscillations task policy in obstacle-free environments that find trajectories with minimized residual load displacement with a specifically designed feature vector for value function approximation. With insights of learning from the cargo delivery problem, we define a set of formal criteria for class of robotics problems where learning can occur in a simplified problem space and transfer to a broader problem space. Exploiting this property, we create a path tracking method that suppresses load displacement. As an extension to tasks in environments with static obstacles where the load displacement needs to be bounded throughout the trajectory, sampling-based motion planning generates collision-free paths. Next, a reinforcement learning agent transforms these paths into trajectories that maintain the bound on the load displacement while following the collision-free path in a timely manner. We verify the approach both in simulation and in experiments on quadrotor with suspended load.